---
---

# Prerequisites

Before starting fine-tuning, please ensure your machine meets the minimum hardware requirements listed in the tables below. The tables show the minimum VRAM (GPU memory) requirements for different models under various configurations.

## CogVideo Series

<table style={{ textAlign: "center" }}>
  <thead>
    <tr>
      <th style={{ textAlign: "center" }}>Model</th>
      <th style={{ textAlign: "center" }}>Training Type</th>
      <th style={{ textAlign: "center" }}>Distribution Strategy</th>
      <th style={{ textAlign: "center" }}>Mixed Precision</th>
      <th style={{ textAlign: "center" }}>Training Resolution (FxHxW)</th>
      <th style={{ textAlign: "center" }}>Hardware Requirements</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td rowspan="6">cogvideox-t2v-2b</td>
      <td>lora (rank128)</td>
      <td>DDP</td>
      <td>fp16</td>
      <td>49x480x720</td>
      <td>16GB VRAM</td>
    </tr>
    <tr>
      <td rowspan="5">sft</td>
      <td>DDP</td>
      <td>fp16</td>
      <td>49x480x720</td>
      <td>36GB VRAM</td>
    </tr>
    <tr>
      <td>1-GPU zero-2 + opt offload</td>
      <td>fp16</td>
      <td>49x480x720</td>
      <td>17GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-2</td>
      <td>fp16</td>
      <td>49x480x720</td>
      <td>17GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-3</td>
      <td>fp16</td>
      <td>49x480x720</td>
      <td>19GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-3 + opt and param offload</td>
      <td>bf16</td>
      <td>49x480x720</td>
      <td>14GB VRAM</td>
    </tr>
    <tr>
      <td rowspan="5">cogvideox-\{t2v,i2v\}-5b</td>
      <td>lora (rank128)</td>
      <td>DDP</td>
      <td>bf16</td>
      <td>49x480x720</td>
      <td>24GB VRAM</td>
    </tr>
    <tr>
      <td rowspan="4">sft</td>
      <td>1-GPU zero-2 + opt offload</td>
      <td>bf16</td>
      <td>49x480x720</td>
      <td>42GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-2</td>
      <td>bf16</td>
      <td>49x480x720</td>
      <td>42GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-3</td>
      <td>bf16</td>
      <td>49x480x720</td>
      <td>43GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-3 + opt and param offload</td>
      <td>bf16</td>
      <td>49x480x720</td>
      <td>28GB VRAM</td>
    </tr>
    <tr>
      <td rowspan="5">cogvideox1.5-\{t2v,i2v\}-5b</td>
      <td>lora (rank128)</td>
      <td>DDP</td>
      <td>bf16</td>
      <td>81x768x1360</td>
      <td>35GB VRAM</td>
    </tr>
    <tr>
      <td rowspan="4">sft</td>
      <td>1-GPU zero-2 + opt offload</td>
      <td>bf16</td>
      <td>81x768x1360</td>
      <td>56GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-2</td>
      <td>bf16</td>
      <td>81x768x1360</td>
      <td>55GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-3</td>
      <td>bf16</td>
      <td>81x768x1360</td>
      <td>55GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-3 + opt and param offload</td>
      <td>bf16</td>
      <td>81x768x1360</td>
      <td>40GB VRAM</td>
    </tr>
  </tbody>
</table>

## CogView Series

<table style={{ textAlign: "center" }}>
  <thead>
    <tr>
      <th style={{ textAlign: "center" }}>Model</th>
      <th style={{ textAlign: "center" }}>Training Type</th>
      <th style={{ textAlign: "center" }}>Distribution Strategy</th>
      <th style={{ textAlign: "center" }}>Mixed Precision</th>
      <th style={{ textAlign: "center" }}>Training Resolution (HxW)</th>
      <th style={{ textAlign: "center" }}>Hardware Requirements</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td rowspan="5">CogView4-6B</td>
      <td>lora (rank128)</td>
      <td>DDP</td>
      <td>bf16</td>
      <td>1024x1024</td>
      <td>30GB VRAM</td>
    </tr>
    <tr>
      <td rowspan="4">sft</td>
      <td>1-GPU zero-2 + opt offload</td>
      <td>bf16</td>
      <td>1024x1024</td>
      <td>42GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-2</td>
      <td>bf16</td>
      <td>1024x1024</td>
      <td>50GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-3</td>
      <td>bf16</td>
      <td>1024x1024</td>
      <td>47GB VRAM</td>
    </tr>
    <tr>
      <td>8-GPU zero-3 + opt and param offload</td>
      <td>bf16</td>
      <td>1024x1024</td>
      <td>28GB VRAM</td>
    </tr>
  </tbody>
</table>
